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when  the  concurrent  task  they  were  involved  in  was  verbal  rather  than  spatial;  conversely,  they  would  be  more  likely  to  use  verbal 
descriptions  when  the  concurrent  task  was  spatial  rather  than  verbal.  These  hypotheses  were  supported  in  a  series  of  experiments 
in  which  we  measured  people’s  time  to  decide  whether  spatial  configurations  displayed  from  different  perspectives  were  the  same 
or  different,  while  they  performed  various  types  of  concurrent  tasks. 
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OBJECTIVE:  This  research  sought  to  1)  develop  a  set  of  experimental  methodologies 
that  could  be  used  to  investigate  spatial  reasoning  strategies  under  timed,  multiple-display 
conditions;  2)  conduct  a  series  of  experiments  examining  the  processing  strategies  people 
use  to  recognized  spatial  configurations  in  uni-  and  multi-tasking  conditions;  and  3)  train 
students  to  generate  research  questions,  develop  experimental  materials,  collect  and 
analyze  data,  and  write  up  results  for  presentation  and  publication. 

APPROACH:  In  our  studies,  participants  viewed  pairs  of  near-photorealistic  3D  scenes 
from  different  points  of  view.  One  of  their  tasks  was  to  say  whether  the  configurations 
depicted  in  the  scenes  were  the  same  or  different.  While  performing  this  task,  participants 
were  also  asked  to  perform  a  secondary  task,  for  example,  keep  in  mind  a  sequence  of 
consonants  or  abstract  symbols.  Tasks  that  draw  on  the  same  set  of  mental  resources  can 
lead  to  selective  interference.  This  paradigm  allowed  us  to  study  not  only  the  strategies 
people  use  in  the  localization  of  objects,  but  also  their  ability  to  flexibly  change  strategies 
as  a  function  of  concurrent  task  demands. 

ACCOMPLISHMENTS :  Since  the  beginning  of  the  project  in  July  2001 ,  we  ran  over 
800  participants  in  15  experiments.  The  work  included  constructing  over  2,600  3D 
images  and  10  relatively  complicated  presentation  programs. 

During  the  course  of  project,  my  students  and  I  examined  the  kinds  of  mental 
processes  people  use  to  recognize  spatial  configurations  from  various  perspectives.  Our 
research  was  based  on  the  idea  that  people  use  at  least  two  main  strategies  in  determining 
whether  two  spatial  configurations  are  the  same  or  different.  One  of  these  strategies  is 
mental  rotation  (e.g.,  Cooper,  1975;  Cooper  &  Shepard,  1973;  Shepard  &  Metzler,  1971; 
Tarr  &  Pinker,  1989,  1990).  An  observer  might  mentally  rotate  a  configuration  to  test 
whether  it  aligns  with  another  configuration  viewed  from  another  perspective.  Another 
strategy  people  might  use  is  to  compare  verbal  descriptions  of  the  two  scenes  (Bethell- 
Fox  &  Shepard,  1988;  Hermer- Vazquez,  Spelke,  &  Katsnelson,  1999;  Simons,  1996). 
Upon  viewing  a  scene,  an  observer  might  covertly  code  certain  aspects  of  the  scene  in 
linguistic  terms.  For  example,  in  a  scene  depicting  a  woman  and  a  couch  (see  Figure  1 
below),  they  might  silently  tell  themselves  “the  woman  is  standing  in  front  of  the  couch.” 
Upon  seeing  the  scene  from  another  perspective,  the  observer  might  form  another  verbal 
description  and  then  compare  it  to  the  former  to  determine  whether  the  scenes  are  the 
same. 

In  one  line  of  experiments,  we  examined  some  of  the  stimulus  conditions  that  lead 
people  to  recognize  spatial  configurations  using  either  mental  rotation  or  verbal 
descriptions.  In  an  important  initial  experiment  (N=40),  we  investigated  how  processing 
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Figure  1.  In  Scenes  B  and  C,  a  simulated  camera  is  panned  90°  from  where  it  was  in  Scene  A.  In 
scenes  A  and  B  the  configurations  are  the  same  while  in  Scenes  A  and  C  they  are  different. 

might  be  influenced  by  the  verbal  discriminibility  of  a  configuration.  Participants  (N=40) 
saw  two  scenes  arranged  side-by-side  on  the  screen.  Each  scene  contained  a  pair  of 
objects  that  included  a  figure  object  (e.g.,  a  woman)  standing  in  front  of  or  behind  a 
ground  object  (e.g.,  a  couch)  (see  Figure  1).  In  half  of  the  scene  pairs,  the  entities  were 
configured  in  the  same  way  while  in  the  remaining  pairs  they  were  configured  differently. 
In  roughly  90%  of  the  scene  pairs,  the  two  scenes  were  rendered  (computer  drawn)  from 
different  perspectives  (e.g.,  0°  &  45°,  0°  &  270°)  while  in  the  remaining  pairs  (-10%) 
they  were  rendered  from  the  same  perspective.  The  images  were  based  on  24  different 
ground  objects.  Half  of  the  participants  saw  configurations  containing  objects  (N=12) 
with  inherent  front  and  back  sides  (see  Figure  2,  panel  A),  while  the  remainder  of  the 
participants  saw  configurations  containing  objects  (N=12)  without  inherent  front  and 
back  sides  (e.g.,  see  Figure  2,  panel  B).  We  measured  people’s  time  to  say  whether  the 
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Figure  2.  Panel  A  shows  configurations  containing  entities  with  inherent  front  and  back  sides  while 
Panel  B  shows  configurations  containing  entities  without  inherent  front  and  back  sides,  along  with 
the  predict  pattern  of  reaction  times  for  the  two  types  of  configurations.  A  flat  slope  across  different 
disparities  in  perspective,  as  shown  in  Panel  A,  suggests  use  of  verbal  descriptions,  while  a  linearly 
increasing  slope,  as  shown  in  Panel  B,  suggests  use  of  mental  rotation. 


configurations  were  the  same  or  different.  As  predicted,  reaction  times  for  the  high- 
verbally  discriminable  scenes  were  equally  fast  from  each  camera  angle — suggesting  use 
of  verbal  descriptions — while  RTs  for  the  low-verbally  discriminable  scenes  increased 
linearly  with  increases  in  the  disparity  in  camera  angle — suggesting  the  use  of  mental 
rotation  (see  Figure  3).  One  small  but  important  departure  from  the  predicted  results 
occurred  in  the  case  of  the  high-verbally  discriminable  scenes.  Participants  responded 
faster  to  zero  differences  in  angular  disparity,  0°,  than  to  the  other  differences  (see  Figure 
3).  We  interpret  this  break  from  the  main  pattern  as  signaling  a  shift  in  processing:  when 
the  scenes  are  precisely  the  same,  people  probably  rely  on  a  superficial,  template-based, 
matching  procedure — rather  than  verbal  descriptions — to  determine  whether  the  scenes 


are  the  same  or  not.  Hence,  people  may  use  verbal  processing  only  when  integrating  their 
knowledge  about  spatial  relations  across  different  points  of  view.  While  the  results  from 
this  experiment  were  consistent  with  our  hypotheses,  they  were  also  consistent  with  the 
possibility  that  people  used  distinctive  asymmetries  in  the  reference  objects  of  the  high- 
verbally  discriminable  scenes  to  match  the  scenes. 

A  second  key  experiment  addressed  this 
limitation.  Participants  (N=80)  were  shown  the 
same  high-verbally  discriminable  scenes  as  | 
before  and  asked  to  say  whether  they  were  the  T 
same  or  different.  In  this  experiment,  | 

however,  half  of  the  participants  saw  pairs  of  | 
scenes  that  differed  in  more  than  one  way:  not 
only  did  the  woman  sometimes  appear  on  the 
opposite  side  (as  before),  she  was  also  sometimes 
turned  1 80-degrees.  Because  the  scenes  could 
differ  in  more  than  one  way,  using  verbal 
descriptions  would  have  required  participants  to 
encode  more  than  one  proposition  (e.g..  The 
woman  is  in  front  of  the  couch  with  her  left 
hand  closer  to  the  couch  than  her  right  hand),  and  hence  form  more  complex  verbal 
descriptions.  As  predicted,  the  reaction  times  (RTs)  for  participants  seeing  pairs  of 
configurations  that  differed  in  more  than  one  way  increased  linearly  with  camera  angle, 
suggesting  use  of  mental  rotation  instead  of  use  of  verbal  descriptions.  However,  the  RTs 
for  participants  seeing  configurations  that  differed  in  only  one  way  were  flat,  suggesting 
use  of  verbal  descriptions.  Thus,  people  tended  to  use  verbal  descriptions  when  those 
descriptions  were  relatively  easy  to  generate;  otherwise,  people  used  the  more  time- 
consuming  process  of  mental  rotation. 

In  a  third  key  experiment  (N=40),  we  provided  further  evidence  for  the  use  of  verbal 
descriptions  by  varying  the  scenes  in  terms  of  the  “goodness”  of  the  spatial  relation 
between  an  object  and  a  person.  In  half  the  scenes  the  person  stood  either  directly  in  front 
of  or  behind  the  object.  In  the  remaining  scenes,  the  person  stood  slightly  to  the  side  of 
these  relations.  As  predicted,  participants’  use  of  verbal  descriptions  was  faster  for  good 
examples  of  these  relations  than  for  not-so-good  examples. 

In  a  fourth  key  experiment  (N=80),  we  found  that  peoples’  recognition  strategies 
changed  under  multi-tasking  conditions.  Participants  saw  the  same  high  verbally 
discriminable  configurations  used  in  the  experiment  described  above.  Half  of  the 
participants  performed  a  concurrent  task — repeating  a  list  of  numbers  heard  through 
earphones — while  the  rest  simply  viewed  the  scenes  as  before.  Verbal  repeating 
(shadowing)  has  been  shown  to  selectively  impair  working  memory  for  linguistic 
information.  As  predicted,  participants  performing  this  secondary  task  tended  to  match 
the  spatial  configurations  using  mental  rotation — as  indicated  by  linearly  increasing 
RTs — while  those  not  performing  the  secondary  task  appeared  to  use  verbal 
descriptions — as  indicated  by  RTs  that  remained  constant  across  different  camera  angles. 
It  is  significant,  however,  in  a  shadowing  task,  that  the  primary  and  secondary  tasks  are 
performed  together. 


Figure  3.  Mean  reaction  times  and  associated 
standard  errors  as  a  function  of  the  differences 
in  angular  disparity  between  the  two  scenes. 


In  a  fifth  key  experiment  (N=64)  we  examined  what  might  happen  when  the  primary 
and  secondary  tasks  were  performed  sequentially.  Participants  saw  the  low-verbally 
discriminable  scenes,  as  described  above,  in  which  both  the  position  and  the  direction  of 
the  person  were  varied.  In  one  condition,  participants  saw  the  scenes  without  a  secondary 
task.  Under  these  conditions,  we  found  that  people  matched  scenes  using  mental  rotation 
just  as  they  had  before.  In  another  condition,  participants  saw  the  scenes  while 
simultaneously  keeping  in  mind  a  small  set  of  consonants.  We  predicted  that  in  this  task 
that  people  would  continue  to  use  mental  rotation.  To  the  contrary,  participants  often 
appeared  to  switch  to  verbal  descriptions  or  to  use  both  verbal  descriptions  and  mental 
rotation,  as  indicated  by  reaction  times  that  were  intermediate  between  those  associated 
with  mental  rotation  and  verbal  descriptions.  This  finding  raises  the  possibility  that  the 
verbal  processes  used  in  the  verbal  secondary  task  primed  people  to  use  verbal  processes 
in  the  scene-matching  task. 

This  conclusion  was  supported  in  a  sixth  key  experiment  (N=64)  in  which 
participants  saw  the  same  scenes  but  performed  a  visual/spatial-processing  task  (e.g., 
keeping  mind  a  matrix  of  squares).  We  predicted  that  such  a  task  would  interfere  with 
their  default  strategy  of  using  mental  rotation.  Just  the  opposite  occurred.  The  processes 
needed  for  the  visual/spatial  secondary  task  appeared  to  prime  the  use  of  yet  another 
visual/spatial  process,  mental  rotation.  These  results,  along  with  others,  suggest  that 
performance  of  two  tasks  can  lead  to  process  priming,  and  not  just  process  interference. 

In  a  seventh  key  experiment  (N=128),  participants  saw  the  high-verbally 
discriminable  configurations,  as  described  above.  In  one  condition,  participants  saw  the 
scenes  without  a  secondary  task.  Under  these  conditions,  we  found  that  people  matched 
the  scenes  using  verbal  descriptions,  just  as  found  in  previous  experiments.  In  another 
condition,  participants  saw  the  scenes  while  keeping  in  mind  a  small  set  of  consonants  (a 
verbal  task)  while  in  another  condition  they  saw  the  scenes  while  keeping  in  mind  a 
matrix  of  squares  (a  spatial  task).  Based  on  the  results  from  Experiments  5  and  6,  we 
predicted  that  the  verbal  secondary  task  would  promote  (process  prime)  use  of  verbal 
descriptions  in  the  primary  task  of  configuration  recognition  while  the  spatial  secondary 
task  would  promote  use  of  mental  rotation.  The  results  were  as  predicted. 

CONCLUSIONS:  Our  findings  suggest  that  the  processing  strategies  people  use  to 
recognize  spatial  configurations  depend  on  their  ability  to  encode  space  in  terms  of 
language  and  the  performance  of  a  secondary  task.  Specifically,  when  verbal  working 
memory  is  not  filled  and  the  spatial  configurations  are  relatively  easy  to  discriminate  in 
terms  of  verbal  descriptions,  people  appear  to  recognize  configurations  using  verbal 
descriptions,  but  when  verbal  working  memory  is  filled — due  to  a  concurrent  task — or 
when  the  spatial  configurations  are  not  easily  discriminated  in  terms  of  verbal 
descriptions,  people  match  spatial  configurations  in  terms  of  mental  rotation. 

SIGNIFICANCE:  There  has  been  relatively  little  research  on  how  people  recognize 
spatial  configurations  from  multiple  points  of  view  while  simultaneously  performing 
other  tasks.  This  research  addresses,  then,  the  spatial  problem-solving  strategies  people 
use  when  tracking  multiple  objects  (e.g.,  ships,  aircraft)  while  also  listening  for  verbal 
messages,  monitoring  warning  lights,  or  repeating  verbal  commands.  Findings  from  this 
research  could  inform  future  tests  of  spatial  ability  and  cognitive  flexibility,  workstation 
design,  or  the  linguistic  codification  of  databases  containing  dynamic  images  (animation 
and  video). 
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